What Hornbeck really means is that he may be a rotten person, but he is on Rachel's side of the argument. He is supporting what Cates said to his class. He sees himself as the rancid butter that nobody likes but he is on her side rather than supporting the other side which is the town's side.
